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Abstract 


Traditional manifold learning algorithms often bear an assumption that the local 
neighborhood of any point on embedded manifold is roughly equal to the tangent 
space at that point without considering the curvature. The curvature indifferent 
way of manifold processing often makes traditional dimension reduction poorly 
neighborhood preserving. To overcome this drawback we propose a new algorith- 
m called RF-ML to perform an operation on the manifold with help of Ricci flow 
before reducing the dimension of manifold. 
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1. Introduction 


In general, traditional manifold learning algorithms can be roughly divided in- 
to two classes: one is to preserve the global geometric structure of manifold, such 
as Isomap [3]; the other one is to preserve the local neighborhood geometric struc- 
ture, such as LLE [2], LEP [4], LPP [7], LTSA [6], Hessian Eigenmap [5] et al. 
Isomap aimed to preserve the geodesic distance between any two high dimension- 
al data points, which can be viewed as an extension of Multidimensional Scaling 
(MDS) [12]. Local neighborhood preserving algorithms approximated manifolds 
with a union of locally linear patches. After the local patches are estimated with 
linear methods such as PCA [8], the global representation is obtained by aligning 
the local patches together. Traditional manifold learning algorithms often bear an 
assumption that the local patch of any point on embedded manifold is roughly 
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equal to the tangent space at that point. Despite the success of manifold learning 
in many applications, there are still several problems remaining. 


e Locally short circuit problem: if the embedded manifold is highly curved, 
the Euclidean distance between any two points is obviously shorter than the 
intrinsic geodesic distance. 


e Intrinsic dimension estimation problem:since tangent spaces are simply 
taken as local patches, the intrinsic dimension of manifold cannot be deter- 
mined by the latter, in particular in case of strongly varying curvature. 


e Curvature sensitivity problem: if the curvature of original manifold is 
especially high at some point, smaller patches are needed for representing 
the neighborhoods around this point. But one may not have as many data 
points as needed to produce enough small patches, especially when the data 
points are sparse. 


Among the above mentioned three problems, the third one is critical. It is the 
background of the other two. To solve this problem is the main target of this paper. 


1.1. Motivation 

In manifold learning one usually maps an irregularly curved manifold to a 
lower dimensional space directly. This is often unpractical since care is not taken 
of the varying geometric structure of the manifold at different points. Ricci flow 
is a very useful tool for evolving an irregular manifold and making it converging 
to a regular one (constant curvature manifold). In this paper we first analysis the 
intrinsic curvature of a manifold at each point and then use Ricci flow to regular- 
ize the metric and curvature of the manifold before reducing its dimension. Since 
Ricci flow preserves local structure of manifold and the Riemannian metric of the 
manifold after Ricci flow is uniform (see section 2), the local relationships among 
data points in the whole process of algorithm are preserved. In the current pre- 
sentation we only consider open Riemannian manifolds with non-negative Ricci 
curvature. Manifold learning with negative Ricci curvature will be discussed in 
our next paper. We call this style of algorithm dynamic manifold learning. This 
paper is just a first attempt in this direction. 


2. Basic Knowledge 


Given data set {7,,%2,--- , £y} C RP, where N is the number of data points 
and D their dimension. We assume that {1),22,---,vy} C R” lie on a d- 
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dimensional Riemannian manifold (M, g) embedded in the high dimensional Eu- 
clidean space R? (d < D), where M is the manifold itself and g its Riemannian 
metric defined as the family of all inner products defined on all tangent spaces of 
M, R? called the ambient space of (M, g). The directional derivative defined on 
a Riemannian manifold is Riemannian connection, represented by V. The cur- 
vature of a d-dimensional Riemannian manifold is represented by a fourth-order 
tensor called the Riemannian curvature tensor Rm (X, Y, Z, W). The trace of the 
Riemannian curvature tensor is a symmetric (0, 2)-tensor called Ricci curvature 
tensor Ric (Y, Z) =trRm(-,Y,-, Z) [1]: 

Regarding Riemannian sub-manifold, the Riemannian curvature tensor is cap- 
tured by the second fundamental form B (X, Y) which is a bilinear and symmetric 
form defined on tangent vector fields X,Y. B (X,Y) is used to measure the dif- 
ference between the intrinsic Riemannian connection V on M and the Riemannian 
connection V on M, where M is embedded into M. The relationship between V 
and V is shown by the Gauss formula [10]: 


VxY =VxY+B(X,Y). (1) 


The corresponding relationship between Rm (X,Y, Z, W) and Rm (X,Y, Z,W) 
is shown by Gauss equation [10]: 


Rm (X,Y, Z,W) = Rm(X,Y,Z,W)—(B(X,W), B (Y, Z))+(B(X, Z) , B(Y,W)). 
_ a (2) 
In this paper M is R?, so Rm (X,Y, Z,W) =0. 


Rm (X,Y, Z,W) = (B(X,W), B(Y,Z)) — (B(X,Z),B(Y,W)). (8) 


Under local coordinate system, the second fundamental form B can be represent- 
ed by [10]: B(X,Y) = eae h° (X,Y) Ea, where Ea (a =d+1,--- ,D) is 
the normal vector field of M and h® (X,Y ) is the second fundamental form coef- 
ficient. 

Ricci flow is defined by the following geometric evolution time dependent 
PDE [14] au = —2Ric;;, where gi; = g (0;,0;). The Ricci curvature Ric (g) is a 
Laplacian of the metric g, making Ricci flow equation a variation of the usual heat 
equation. In the time interval t € J the Riemannian metric g (t) satisfies the met- 
ric equivalence condition e~?“'g (0) < g(t) < e?'g (0) [14], where |Ric| < C. 
So the relative geodesic distance between arbitrary two neighborhood points on 
M is consistent under the Ricci flow. In [11] researchers have worked out that the 
Riemannian manifold of dimension d > 4 can be transformed to a sphere under 
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the spherical condition with the sectional curvature K satisfied mark < 4 every- 
where. 


3. Algorithm 


In practice, it is difficult to analyze the global structure of a nonlinear mani- 
fold, especially when there is no observable explicit structure. In this paper we 
decompose the embedded manifold into a set of overlapping patches and apply 
Ricci flow to these overlapping patches independently of each other to avoid sin- 
gular points. The global structure of deformed manifold under Ricci flow can be 
obtained from the deformed local patches with a suitable alignment. 

Our algorithm RF-ML is mainly divided into six steps: 


1. Find a local patch (neighborhood) U;,2 = 1,2,--- ,N for each data point 
zi, t = 1,2,--- , N by using the K-nearest neighbor method. 


2. Construct a special local coordinate system on every point x;. In the neigh- 
borhood U, we estimate the local patch information of x; with a covariance 
matrix C;, C; = Peek (£k — T)” (£k — Ti), where 7; is the mean vec- 
tor of the K-nearest data points. The first d eigenvectors (e1, €2, +- , eq) 
with maximal eigenvalues of C; form a local orthonormal coordinate sys- 
tem of x; on U;. The last D — d eigenvectors (e441, +- , €p) form a local 
orthonormal coordinate system of normal space. 


3. The intrinsic dimension d of local patches in this algorithm is determined 
by the number of the 95% principal components. In practice due to the 
different curvature of local patches, the local dimension d; of each patch U; 
may be in practice not all the same. We choose d = maz{dj, dz,--- , dy}. 
If d = D, stop the algorithm. The manifold’s dimension is not reducible. 
If d < D, continue the algorithm. 


4. Construct Ricci flow equations on local patches and then let the overlapping 
patches U;,72 = 1,--- , N flow independently into local spherical patches 
Y;,2 = 1,2,--- ,N with constant positive Ricci curvature C. Details will 
be shown below. 


5. Align the discrete sphere patches Y;,2 = 1,2,--- , N into a global subset 
P of a sphere with positive curvature C, where P C RP. p; € P is the 
corresponding representation of x;. Details will be shown below. 
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6. Reduce the dimensionality of the subset P using traditional manifold learn- 
ing algorithms, where the distance metric between arbitrary two points on 
P is the metric of the sphere other than the Euclidean. The d-dimensional 
representations are { 21, 22,--- , zn} € Re, 


Step 4) above aims to minimize the following energy function such that the 
Ricci curvature at every point converges to a constant curvature C. 


E (Ric) = f Ric- C||’dM = ? |Ric(X,) — C}. (4) 


In order to obtain the minimum solution of the curvature energy function, we cal- 
culate the solution step by step with the help of Ricci flow. The Ricci flow defined 
on the discrete data points {x£1, £2,- , £y} € R? is constructed as follows: 

Suppose the local coordinates of any point x; € U; under local coordinates 
(£i; e1, €2,°** ,€q,*** , €p) are represented as (xt, £ĉ,- -- £P). The first d coor- 
dinates can be seen as the local natural parameters. A smooth representation of 
local patch U; under the local coordinate system is described by: 


f Ui) = (a ar +4 KN aia (a), a? ,< 9a") + a (£1, a 2 a) (5) 


where (xt, e a) is a coordinate chart at U;. 
We use least square method to approximate the analytic functions f%,œa = d+ 
1,--- ,D. In order to guarantee the curvature operator on each patch being sat- 


isfied the spherical conditions as presented in section 2, we choose second-order 
elliptic polynomial functions to approximate the structures of local patches. Un- 
der the local coordinates of U;, the corresponding d tangent vector bases on x; are 


given by os (zi), Bs (tases gh (x;). 


of ofat ðf? 
a a,j (ei) = (0... 1l,- 0, E e) Fe). 


Then the local Riemannian metric tensor is G; = [g;z], Ir = (gf (Bi); H (x;)}. 


(6) 
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The second fundamental form coefficient is hgy, = soir Bak According to Gaussian 
equation and the definition of Ricci flow mentioned in section 2, the Ricci flow 
equation defined on X; is: 


a af əf T n 3 3 a fe . or fo _ ar fe . ar fo 
Ot \ OxI Oxk E Əxl!ðx! ðxiðxzk Oax'Oxk OAx'Oxk A 


a=d+1 l=1 
(7) 


In order to solve the Ricci flow equation, we need to discretize the differential 
operators in each local patch U;: 


fit = 2 aS ( aft eft aft aft re 
l=1 


Ori Ones Arde! ðriðxzİi ~~ Ax! Oxi Ox'Oxs Oxi 
ə fe ð f T 
Ia = jk 4 al a aa ) i (8) 


eee 3 or pitt gfi E or pitt gfi 
jk Əðx'ðx! Ox*Oxi Əx'ðxİ Ox!Ox* j` 


l=1 


By optimizing these update equations we have Rọ —> C, where C is a non- 
negative constant. 

In step 5) above, after the Ricci flow converges at each X;, the overlapping lo- 
cal patches {U;, U2,--- , Uyn } are mapped to a set of discrete local spherical patch- 
es {Y1, Yo,--- , Yn}. We denote the global coordinates of {Y1, Y2,--- , Yn} with 
{P\, Po,--- , Pyu}. The relationship between {Yi,--- , Yy} and {P,,--- , Py} is 
a set of global alignment maps. which shift the discrete local spherical patches to 
a global subset of a sphere. In each local patch Y; we have: 


pi, = Pit Ty, +e i=1,---, Nj =1,---,K, (9) 


where T; is the unit of orthogonal transformation and g the reconstruction error, 
Yy € Yi, Di, € Pi 

In order to obtain the optimized global affine transformation, we minimize the 
global reconstruction error matrix: 


N 
1 
min X` ||P, (1 = xe) Ty. (10) 
i=1 


Minimizing the above least square error is equal to solve the following eigenvalue 
problem: 

B = SWWT ST, (11) 
where S = [Si, So, ans Sy], PS; = P;, W = diag (Wi, Wop, a7 4 Wyn), 
W; = ( I— pee”) (1 — YÌY,) and Yl the generalized inverse matrix of Y;. 
Decomposing matrix B using SVD method, we obtain B = UAU~!. The optimal 
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Figure |: The intuitive process of algorithm RF-ML. The sub-images from left to right are respec- 
tively: input data points distributed on manifold M; the overlapping patches; overlapping patches 
flowing into discrete spherical patches using Ricci flow; alignment of spherical patches to a subset 
of a sphere. On the bottom is the representation of data points in low dimensional Euclidean space 


data set P that we finally need are distributed on a sphere with curvature C. So we 
need to give a set of constraints P,P,’ = g;,i = 1,2,--- , N. Under these con- 
straints, we rewrite matrix B as: B = QRAR™Q’, where R is a diagonal matrix, 


Ri = C+ 9/02 UZ, i = 2,--+, D +1, the rest Rix = 1,k # 2, ,D +1. 
The obtained 2”¢ to D + 1%* columns of Q are the optimal data set P, which are 
distributed on a sphere with curvature C’. 


An intuitive representation of algorithm RF-ML is shown in Figure 1. 


4. Experiment 


We compare our method with traditional manifold learning algorithms (PCA, 
Isomap, LLE, LEP, Diffu-Map, LTSA) on four sets of three dimensional data 
from [13] including Swiss Roll, Sphere, Ellipsoid and Gaussian . The objective 
of this comparison is to map each data set to two dimensional space and then to 
analyze the neighborhood preserving ratio (NPR) [9] of different algorithms. The 
neighborhood preserving ratio (NPR) [9] is defined as follows: 


T 
NB= KN IV (a) (N (a) | (12) 


where N (zx;) is the set of K-nearest sample subscripts of x;, and M (z;) is the set 
of K-nearest sample subscripts of z;. | - | represents the number of intersection 
points. 


Table 1: Neighborhood Preserving Ratio. In this experiment, we fix the neighborhood-size param- 
eter K = 10. 


Methods PCA | Isomap | LLE LEP | Diffu-Map | LTSA | RF-ML 


Swiss Roll | 0.5137 | 0.8594 | 0.6187 | 0.3981 0.4403 | 0.6121 | 0.8594 


Ellipsoid | 0.4399 | 0.6914 | 0.6205 | 0.7506 | 0.5965 0.4390 | 0.8702 


Sphere 0.4815 | 0.6467 | 0.5213 | 0.7720 | 0.5010 | 0.5465 | 0.8684 


Gaussian | 0.9969 | 0.9261 | 0.9359 | 6406 0.9848 | 0.9970 | 0.9909 


Table 1 shows that for all but one dataset, the NPR of RF-ML is the best one 
among all algorithms. Using our algorithm we can analysis that Swiss Roll is 
a locally flat two-dimensional manifold. Its data structure is unchanged under 
Ricci flow. As for the Sphere dataset, its Gauss curvature is a unique constant ev- 
erywhere. Regarding Ellipsoid dataset and Gaussian dataset, note that the Gauss 
curvatures at different points are not always the same. When using RF-ML to 
reduce their dimension, the NPR of Ellipsoid dataset is 87.95%, 34.28%, 66.58%, 
12.49%, 73.33%, 58.90% respectively better over the other six traditional mani- 
fold learning algorithms. For Gaussian dataset, due to the characteristics of the 
dataset distribution the NPRs of PCA and LTSA are quite high. Compared with 
the six algorithms, the NPR of our method is not bad and also quite high. These 
results clearly demonstrate that our curvature aware algorithm RF-ML is more sta- 
ble and the Ricci flow process preserves better the local structure of data points. 


5. Conclusions and Future Work 


Whether the manifold structure is uncovered exactly or not impacts the learn- 
ing results directly. Traditional manifold learning algorithms do not consider the 
varying curvature at different points of the manifold. Our method aims to excavate 
the power of Ricci flow and to use it to dynamically deform the local curvature 
to make the manifolds curvature uniform. The extensive experiments have shown 
that our method is more stable compared with other traditional manifold learning 
algorithms. One limitation of our algorithm is that RF-ML works only for man- 
ifolds with non-negative curvature. We will discuss the applicability of RF-ML 
algorithm to manifolds with negative Ricci curvature manifold in our next paper. 
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